Papers with automated reasoning
SCRIPTMIND: Crime Script Inference and Cognitive Evaluation for LLM-based Social Engineering Scam Detection System (2026.eacl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promise in identifying deception, but their cognitive assistance potential remains underexplored. |
| Approach: | They propose a framework for LLM-based scam detection that bridges automated reasoning and human cognition. |
| Outcome: | The proposed framework outperforms GPT-4o in the Korean scam detection and phone scam simulations. |
ARQA: A Benchmark for Grounded Table–Text QA in Enterprise Annual Reports (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing QA benchmarks focus on retrieval or single-modality reasoning . annual reports are a company's definitive record of performance . |
| Approach: | They propose an annual report QA benchmark that compares QAs with lookups, arithmetics, and insights. |
| Outcome: | The proposed benchmarks show strong factual retrieval but persistent weaknesses in grounded arithmetic and causal reasoning. |
What Action Causes This? Towards Naive Physical Action-Effect Prediction (P18-1)
Copied to clipboard
| Challenge: | a new task on naive physical action-effect prediction addresses the relationship between concrete actions and their effects on the state of the physical world as depicted by images. |
| Approach: | They propose a task that harnesses web image data to facilitate action-effect prediction. |
| Outcome: | The proposed approach harnesses web image data through distant supervision to facilitate learning for action-effect prediction. |
Can Language Models Learn Embeddings of Propositional Logic Assertions? (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for automating reasoning can no longer be used for natural language tasks. |
| Approach: | They propose to use transformer-based language models to reason about knowledge expressed in natural language rather than using LMs to perform reasoning directly. |
| Outcome: | The proposed approach is feasible to some extent, but lacks robustness. |
Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Progress in the task of Critical Questions Generation has been hindered by the lack of suitable datasets and automatic evaluation standards. |
| Approach: | They propose a comprehensive approach to support the development and benchmarking of systems for this task. |
| Outcome: | The proposed approach supports the development and benchmarking of systems for this task. |
LAMBADA: Backward Chaining for Automated Reasoning in Natural Language (2023.acl-long)
Copied to clipboard
| Challenge: | Recent advances in automated reasoning with natural text suffer from a combinatorial explosion of the search space and high failure rates for problems requiring longer chains of reasoning. |
| Approach: | They propose a Backward Chaining algorithm that decomposes reasoning into four sub-modules and implements it by few-shot prompted LLM inference. |
| Outcome: | The proposed algorithm achieves sizable accuracy boosts over state-of-the-art forward reasoning methods on two challenging logical reasoning datasets. |
CHIRON: Rich Character Representations in Long-Form Narratives (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing systems for character representation have simplified the problem of representing complex characters via graphs and brief character descriptions. |
| Approach: | They propose a ‘character sheet’ based representation that organizes and filters textual information about characters. |
| Outcome: | The proposed representation organizes and filters textual information about characters and is better and more flexible than previous models. |
RLMEval: Evaluating Research-Level Neural Theorem Proving (2025.findings-emnlp)
Copied to clipboard
| Challenge: | RLMEval evaluates large language models for research-level neural theorem proving and proof autoformalization . the best model achieves only a 10.3% pass rate on existing benchmarks . |
| Approach: | They propose a new evaluation suite for large language models . it evaluates research-level theorems from real-world Lean formalization projects . |
| Outcome: | RLMEval evaluates research-level theorems from real-world Lean formalization projects. |
Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite advances in large language models, their application to misinformation detection remains hindered by issues of logical inconsistency and superficial verification. |
| Approach: | They propose a multi-agent debate framework that reformulates misinformation detection as a structured adversarial debate based on fact-checking workflows . |
| Outcome: | The proposed framework enables iterative refinement of evidence while improving decision transparency. |